Background modeling for generative image models

Keywords: Generative models Face model Face analysis Morphable Model Bayesian model Implicit background models a b s t r a c t Face image interpretation with generative models is done by reconstructing the input image as well as possible. A comparison between the target and the model-generated image is complicated by the fact that faces are surrounded by background. The standard likelihood formulation only compares within the modeled face region. Through this restriction an unwanted but unavoidable background model appears in the likelihood. This implicitly present model is inappropriate for most backgrounds and leads to artifacts in the reconstruction, ranging from pose misalignment to shrinking of the face. We discuss the problem in detail for a probabilistic 3D Morphable Model and propose to use explicit image-based background models as a simple but fundamental solution. We also discuss common practical strategies which deal with the problem but suffer from a limited applicability which inhibits the fully automatic adaption of such models. We integrate the explicit background model through a likelihood ratio correction of the face model and thereby remove the need to evaluate the complete image. The background models are generic and do not need to model background specifics. The corrected 3D Morphable Model directly leads to more accurate pose estimation and image interpretations at large yaw angles with strong self-occlusion. A human face in a typical image is surrounded by arbitrary background. In Analysis-by-Synthesis settings, generative, para-metric face models such as Active Shape Models, Active Appearance Models or Morphable Models, serve to reconstruct the input face as well as possible [5,4,2]. Depending on its parameter values, the model produces a synthetic image which is then compared to the input image through its likelihood under the model for a given set of parameter values. Since the face only occupies a part of the input image and it can appear in front of any background, one avoids to include background into the model likelihood. Consequently , the likelihood considers only the visible parts and ignores the rest of the image. But as we show in this article, even though background is ignored, it is still present in the model likelihood in the form of an implicit and usually wrong background model. The wrong background model leads to a strong preference for background over the face. Wherever possible, the optimization algorithm will try to reduce the support of the face. This leads to …